Model-mediated Teleoperation with Predictive Models and Relative Tracking

This paper presents a model-mediated approach for teleoperation with haptic feedback in the presence of time delays on the order of seconds. The target application for the control scheme is teleoperation of robotic manipulators for space systems in geosynchronous orbit. Previous work in model-mediated teleoperation allowed operators to interact with a virtual model of the remote robot and environment, where the remote robot follows the operator’s commands after a delay and the virtual model is updated when the remote data is available. Our approach adds predictive models, mediated command execution, and a dynamic slave model. A singledegree-of-freedom experiment using a simulated robot and environment demonstrate improvements in the control of remote robot position and environment contact forces, in comparison to previous approaches. INTRODUCTION Teleoperation of robotic manipulators in outer space has been the subject of research for many decades. For a summary of early work, see Sheridan [1]. More recently, experimental telerobotic systems have been demonstrated in space [2]. However, there remain many unmet challenges in safe and effective manipulation of teleoperated robots under the large time delays inherent in transmission of data between earth and the robot [2]. This paper addresses human-in-the-loop control of a robotic system with communication delays on the order of seconds, motivated by the application of manipulation and repair of satellites in geosynchronous orbit [1]. Effective haptic-feedback teleoperation of these systems requires that: (1) the feedback and control account for the time delay to prevent the need for operators to use a move-and-wait strategy; (2) the visual and haptic feedback necessary to perform dexterous manipulation is provided to the operator; and (3) the remote robot interacts with its environment safely and stably [5]. Previous work has examined the use of predictive displays for visual feedback in remote robot control [2]. Here we focus on a prediction of the environment for the purpose of rendering haptic feedback. We present a telerobotic control framework based on model-mediated teleoperation. The model-mediated approach we use extends the work of Mitra and Niemeyer [3] and Willaert et al. [4], who demonstrated stable haptic feedback under large time delays. Our approach improves interaction with moving objects through the use of a predictive model and a method of executing the operator commands in the remote environment. We also provide the operator with feedback of the dynamic state of the slave manipulator. A single-degree-offreedom (DOF) system is used for discussion and simulations; however, the presented concepts can apply to higher-DOF systems. Simulation results demonstrate the benefits of the predictive model and command execution for interaction with moving objects in the environment. BACKGROUND Control of teleoperation with time delay has been the subject of much past research. An extensive comparison of some existing methods can be found in Arcara and Melchiorri [5]. These previous techniques, mostly based on passivity and scattering theory, are designed to enable direct bilateral teleoperation for relatively small delays of up to a few hundred milliseconds [5]. The application of such controllers results in poor performance or instability in the presence of larger delays. Model-mediated teleoperation has been demonstrated to enable stable haptic interaction for delays on the order of seconds [3-4]. In this method, the master robot interacts with a model of the slave robot and environment, in contrast with direct teleoperation, where the master directly commands the real slave robot. The environment model and parameters may be updated using sensor data from the slave robot. Modelmediated teleoperation has primarily taken two forms in the literature: one form focuses on updating geometric properties [3, 4] and the other focuses on updating dynamic model properties [6]. Our approach uses geometric model-mediated teleoperation, extending [3, 4]. MODEL-MEDIATED TELEOPERATION FRAMEWORK Model-mediated teleoperation is comprised of local teleoperation between a master and virtual slave acting within a virtual environment, and control of a remote robot based on operator commands and remote sensory feedback, as shown in Figure 1. The communication between the local operator/virtual environment and the remote slave robot is subject to time delays on the order of seconds. The bilateral teleoperation takes place solely between the operator and the local environment with minimal time delay. Two key components of model-mediated teleoperation are the estimation and update of the local environment model and the remote implementation of operator commands by the slave robot. The local model must update based on sensor data from the remote slave robot because the remote environment will change. Additionally, the slave should not necessarily directly track the delayed master position. Instead, the operator’s commands, along with the current state of both the local environment model and true remote environment are used to determine the command to the slave. The following subsections will discuss each component individually. For clarity, a singleDOF system is presented. Bilateral Control of the Virtual Slave The bilateral control between the master and the virtual slave is a position-exchange controller. The virtual slave tracks the master device using, for example, proportional-derivative (PD) control, Fvs = kpm(xm − xvs) + kdm(vm − vvs), (1) where Fvs is the force on the virtual slave, kpm and kdm are the PD gains for the local controller, and xm, xvs, vm, and vvs are the positions and velocities of the master device and virtual slave. The force feedback to the master is the negative of the force applied to the virtual slave, Fm = −Fsv. Virtual Slave Model A model of the slave robot is used to feed back the dynamics of the slave to the operator. This is a departure from [3], which used a haptic proxy that did not reflect the dynamic properties of the remote slave. The virtual slave model provides a visual predictive display similar to those used previously for time-delayed teleoperation and also provides kinesthetic feedback of the slave dynamics. Thus, a slave robot with a slow maximum velocity will apply a force to slow the operator motion. This is particularly important when the slave robot has relatively low bandwidth or a velocity limit. The slave model used here is a mass-damper model, mvs?̈?vs + bvs?̇?vs = Fvs − Fvw , (2) where mvs and bvs are the mass and damping coefficients and Fvw is the force of the virtual environment on the slave. Environment Model The rigid wall in the virtual environment is a Kelvin-Voigt model with discontinuous contact [6], Fvw = { kvwΔx + bvwvvs, xvs ≥ xvw , vvs ≥ 0 kvwΔx, xvs ≥ xvw , vvs < 0 0, otherwise where Δx = (xvs − xvw), (3) where kvw and bvw are the stiffness and unilateral damping of the virtual wall, xvw is the location of the virtual wall, and Δx is the penetration of the virtual slave into the virtual wall. Since the objects in the environment are rigid, the stiffness and damping are given high values that maintain stability during contact. Unlike in [6], it is not assumed that the virtual objects represented by these walls have static geometries. Therefore, the geometry of the object must be estimated and updated in the model as surfaces are discovered and tracked by remote sensing. Environment Model Update We focus here on updating the geometric properties of the environment model. It is assumed that an a priori model of the environment is available, but there is no a priori knowledge of the location of objects in the environment. It is assumed that acquired data includes visual detection of object locations that can be used along with the prior knowledge of the environment to form an estimate of the environment’s geometric properties, which will be continually updated as the objects in the environment change location and orientation. Because of communication delays in both the forward and feedback channels, the estimate of the environment should include a predictive component to enable the operator to interact with a changing environment. FIGURE 1. BASIC STRUCTURE OF MODEL-MEDIATED CONTROL FOR SPACE TELEOPERATION. Here, the simplest prediction model is used to demonstrate the benefit of prediction in model-mediated teleoperation. Given a measured object location, an estimate of the object velocity is used to extrapolate forward in time by the total delay to estimate the position of the object at the time the operator’s command will reach the remote robot, xfdbk = xw + Td?̇?w , (4) where xw is the true environment wall position, Td is the round trip time delay, and xfdbk is the predicted wall location. In addition to providing continuous updates of the environment model, it is also important that the local model be able to handle discontinuous changes in the environment feedback. The mediation of discontinuous changes in feedback information of the remote environment is central to the concept of model-mediated teleoperation as proposed in [3] and further described in [4]. Our approach builds on their work. Under normal circumstances, when the state of the environment is known and the virtual slave is either in free space or in contact with the virtual environment, then the virtual wall displayed will be equal to the predicted feedback from (4),